Neural networks with nonlinear synapses and a static noise.

where N is the total number of neurons. The synapses defined by Eqs. (2) and (3) are linear: The changes M;J induced by the addition of a new pattern have a fixed magni«de, IJJJ p'Q'/N + 1/N, which is independent of previous mcHlor1cs. The statistical mechanics of Hopfield's model, Eqs. (1)-(3), has been recently studied2 in the limit of N ~ ~. Three classes of metastable states have been found: retrieval states, spin-glass states, and mixture states. Retrieval states, each of which has a large overlap with a single pattern, exist when a~p/N &a,=0.14. These states are the most important ones for retrieval of memory. The overlap of each state with the corresponding pattern is m I/N g g/'S;. Its value a.t maximum capacity (and zero temperature) is m =0.97. The shght reduction of m from unity is due to the sma11 internal static noise vvhich is generated by the random overlaps among the patterns. As a decreases, 1 —rn decreases very rapidly yielding full retrieval of memory (trt 1) as a~0. For any finite a, there is also a spin-glass state which has zero overlaps with the patterns. In addition, at sufficiently small a, mixture Neural networks which exhibit features of learning and associative memory can be modeled' by a system of Ising spins with an energy function,